Flashes (electrical flashes or strobes) are used as auxiliary light sources during camera photography. In recent years, DSCs (Digital Still Cameras) are rapidly spreading, and flash photography is often performed with DSCs as well. Flashes can be used to perform various kinds of photography such as fill-in light (a technique for weakening a shadow which appears extremely strongly on the face of a person or the like), backlight compensation (a technique for preventing the face of a person from losing shadow detail when the person standing with the sun behind is photographed), catch light (a technique for beautifully photographing the pupil of an eye with a twinkling “light spot” reflected in the pupil), and daylight synchronized flash (daylight synchro, a technique which uses flash as auxiliary light rays in the daytime or evening). On the other hand, when flash photography is performed, there is a case where color balance is impaired or loss of highlight detail occurs. One object of the present invention is to provide appropriate means capable of correcting undesirable phenomena which occur during flash photography.
In general, in digital cameras, white balance (WB) is performed so that an image of a white subject can be taken in white light. For example, white balance adjustment is performed in such a manner that when an image is to be taken in a light component environment, such as natural light, illumination light or flash (strobe, in which the color temperature of light irradiated toward a subject is high and blue (B) light is strong, sensitivity to blue light is suppressed, whereas when an image is to be taken in a light component environment in which the color temperature of light irradiated toward a subject is low and red (B) light is strong, sensitivity to red light is suppressed and sensitivity to blue (B) light is made relatively high.
White balance adjustment generally executes adjustment processing in which adjustment parameters are set according to light sources to be used during image capture. For example when image capture with flash is performed, white balance adjustment is performed in accordance with parameters corresponding to light components of flash light used.
However, if image capture with flash is performed when there is ambient light other than flash light, two kinds of lights, i.e., flash light and ambient light, are irradiated to a subject and reflected light therefrom reaches the image pickup element of the camera, whereby image capture is performed. In the case of this image capture, if white balance adjustment is carried out according to flash light, the section of the subject which is irradiated with a large amount of flash light is adjusted to natural color. However, if white balance adjustment according to parameter settings matched to the light components of flash light is performed on an area which is out of the reach of flash light and is captured in reflected light of only ambient light, for example, a background image area, appropriate white balance adjustment is not executed and the area is outputted as an area having unnatural color.
Conversely, if white balance adjustment matched to a background section, i.e., white balance adjustment based on the assumption that the image is taken with only ambient light, is executed on the entire captured image, the section irradiated with a large amount of flash light is adjusted to unnatural color.
To cope with this problem, several constructions have been proposed. For example, Patent Document 1 (Japanese Patent Application Publication Hei8-51632) discloses a construction which acquires an image taken without flash and an image taken with flash, divides each of these two captured images into blocks, making a comparison between their luminance values in units of one block, and performs different white balance adjustment on each block of the image taken with flash on the basis of the result of the comparison between the luminance values.
During white balance adjustment, any one of white balance adjustment matched to flash light, white balance adjustment matched to light intermediate between flash light and ambient light and white balance adjustment matched to ambient light is selected and executed. However, this construction needs to perform processing in units of blocks, resulting in problems such as the occurrence of block distortion and the problem that no correct processing can be effected when a subject moves.
Patent Document 2 (Japanese Patent Application Publication JP-A-2000-308068) discloses the following processing construction. Namely, an image is taken with flash with a fully open aperture and a short exposure time, and after that, an image is taken without flash under the originally intended exposure conditions. The former and the latter are respectively called a first image and a second image. In addition, in the first image, pixels having not less than a predetermined level are registered as a main subject area, and the other pixels are registered as a background area. After that, the first image is subjected to white balance adjustment matched to flash light and the second image is subjected to white balance adjustment matched to ambient light, and the main subject area of the first image and the background area of the second area are combined to generate a final recorded image.
However, in this construction, it is impossible to correctly effect white balance adjustment of a subject irradiated with both ambient light and flash light.
Patent Document 3 (Japanese Patent Application Publication JP-A-2000-307940) discloses a construction in which image shake detection means is added to the above-mentioned construction of Patent Document 2. In this construction, if it is determined that a shake has occurred, the above-mentioned first image is used as a recorded image without modification and the processing of combining the first image and the second image is not executed. Accordingly, if a shake is detected, unnaturalness due to the difference in color temperature between flash light and ambient light cannot be solved.
Patent Document 4 (Japanese Patent Application Publication Hei8-340542) discloses a construction which performs division on the luminance of each pixel of an image taken with flash and the luminance of the corresponding pixel of an image taken without flash to find the contribution of flash light, and performs white balance adjustment on the image taken with flash, on the basis of this contribution.
In this construction, an image taken with mixed reflected lights of flash light and ambient light is simply interpolated with white balance parameters for flash light and ambient light on the basis of the contribution of flash light, thereby generating a final image. However, if a physical reflection model of light is taken into account, components originating from flash light and components originating from ambient light should be independently processed, and it is impossible to generate an optimum result image merely by processing the image taken with mixed reflected lights of flash light and ambient light.
[Patent Document 1]
Japanese Patent Application Publication Hei8-51632
[Patent Document 2]
Japanese Patent Application Publication JP-A-2000-308068
[Patent Document 3]
Japanese Patent Application Publication JP-A-2000-307940
[Patent Document 4]
Japanese Patent Application Publication Hei8-340542